DatadogをインストールしたECSのCFnテンプレート
DatadogでECSを監視した検証環境がほしく、AWS環境構築とDatadogの設定をCFnで実施しましたのでそのアウトプットです。ECSのデータプレーンはEC2、Fargateでそれぞれ実施しています。
前提
- Datadogアカウントが作成済みであること(フリートライアル有り)
- AWSインテグレーションが実施済みであること
- ECRにApachのイメージ(mod_statusモジュールを使用したExtendedStatusが有効化されていること)が保存されていること
ECS on EC2
ECS on EC2では、コンテナインスタンス上でDatadog Agent(Datadog Docker Agent)のタスクを常駐させる構成となります。これにより、クラスター内すべてのコンテナを監視することが可能になります。以下のような構成です。
公式ドキュメントを参考に、上記構成を構築するCFnテンプレートを作成しました。テンプレートを利用する際は、パラメータにDatadogのAPI キーと、ECRに保存した監視対象となるコンテナイメージを指定してください。
sample-ecs-ec2-datadog.yml
AWSTemplateFormatVersion: 2010-09-09 Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: Common Configuration Parameters: - SysName - Env - Label: default: ECS App Container Setting Parameters: - AppContainerImageName - Label: default: ECS Container Instanse Setting Parameters: - ContainerInstanseAMI - ContainerInstanceType - Label: default: ECS Datadog Container Setting Parameters: - DatadogApiKey - DatadogSite Parameters: SysName: Type: String Default: test Env: Type: String Default: prd AllowedValues: - prd - dev AppContainerImageName: Type: String Default: xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/xxxxx:latest ContainerInstanseAMI: Description: AMI ID Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id> Default: /aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id ContainerInstanceType: Description: ECS Container Instanse EC2 instance type Type: String Default: t3.small DatadogApiKey: Type: String NoEcho: true DatadogSite: Type: String Default: datadoghq.com Resources: # ------------------------------------------------------------# # VPC # ------------------------------------------------------------# Vpc: Type: AWS::EC2::VPC Properties: CidrBlock: 10.0.0.0/16 EnableDnsSupport: true EnableDnsHostnames: true Tags: - Key: Name Value: !Sub ${SysName}-${Env}-vpc Igw: Type: AWS::EC2::InternetGateway Properties: Tags: - Key: Name Value: !Sub ${SysName}-${Env}-igw IgwAttach: Type: AWS::EC2::VPCGatewayAttachment Properties: VpcId: !Ref Vpc InternetGatewayId: !Ref Igw SubnetPublicA: Type: AWS::EC2::Subnet Properties: VpcId: !Ref Vpc CidrBlock: 10.0.1.0/24 AvailabilityZone: ap-northeast-1a MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub ${SysName}-${Env}-subnet-pub-1a SubnetPublicC: Type: AWS::EC2::Subnet Properties: VpcId: !Ref Vpc CidrBlock: 10.0.2.0/24 AvailabilityZone: ap-northeast-1c MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub ${SysName}-${Env}-subnet-pub-1c SubnetProtectA: Type: AWS::EC2::Subnet Properties: VpcId: !Ref Vpc CidrBlock: 10.0.3.0/24 AvailabilityZone: ap-northeast-1a MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub ${SysName}-${Env}-subnet-pro-1a SubnetProtectC: Type: AWS::EC2::Subnet Properties: VpcId: !Ref Vpc CidrBlock: 10.0.4.0/24 AvailabilityZone: ap-northeast-1c MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub ${SysName}-${Env}-subnet-pro-1c RouteTablePublic: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref Vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-rtb-pub RouteTableProtectA: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref Vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-rtb-pro-1a RouteTableProtectC: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref Vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-rtb-pro-1c RouteIgw: Type: AWS::EC2::Route Properties: RouteTableId: !Ref RouteTablePublic DestinationCidrBlock: 0.0.0.0/0 GatewayId: !Ref Igw AssocationPublicA: Type: AWS::EC2::SubnetRouteTableAssociation Properties: SubnetId: !Ref SubnetPublicA RouteTableId: !Ref RouteTablePublic AssocationPublicC: Type: AWS::EC2::SubnetRouteTableAssociation Properties: SubnetId: !Ref SubnetPublicC RouteTableId: !Ref RouteTablePublic NatGatewayEipA: Type: AWS::EC2::EIP Properties: Domain: vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-eip-ngw-1a NatGatewayEipC: Type: AWS::EC2::EIP Properties: Domain: vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-eip-ngw-1c AssocationProtectA: Type: AWS::EC2::SubnetRouteTableAssociation Properties: SubnetId: !Ref SubnetProtectA RouteTableId: !Ref RouteTableProtectA AssocationProtectC: Type: AWS::EC2::SubnetRouteTableAssociation Properties: SubnetId: !Ref SubnetProtectC RouteTableId: !Ref RouteTableProtectC NatGatewayA: Type: AWS::EC2::NatGateway Properties: SubnetId: !Ref SubnetPublicA AllocationId: !GetAtt NatGatewayEipA.AllocationId Tags: - Key: Name Value: !Sub ${SysName}-${Env}-ngw-1a NatGatewayC: Type: AWS::EC2::NatGateway Properties: SubnetId: !Ref SubnetPublicC AllocationId: !GetAtt NatGatewayEipC.AllocationId Tags: - Key: Name Value: !Sub ${SysName}-${Env}-ngw-1c NatGatewayRouteA: Type: AWS::EC2::Route Properties: RouteTableId: !Ref RouteTableProtectA DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: !Ref NatGatewayA NatGatewayRouteC: Type: AWS::EC2::Route Properties: RouteTableId: !Ref RouteTableProtectC DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: !Ref NatGatewayC NetworkAclPublic: Type: AWS::EC2::NetworkAcl Properties: VpcId: !Ref Vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-nacl-pub NetworkAclEntryPublicIngress: Type: AWS::EC2::NetworkAclEntry Properties: CidrBlock: 0.0.0.0/0 Egress: false NetworkAclId: !Ref NetworkAclPublic Protocol: -1 RuleAction : allow RuleNumber : 100 NetworkAclEntryPublicEgress: Type: AWS::EC2::NetworkAclEntry Properties: CidrBlock: 0.0.0.0/0 Egress: true NetworkAclId: !Ref NetworkAclPublic Protocol: -1 RuleAction : allow RuleNumber : 100 NetworkAclProtect: Type: AWS::EC2::NetworkAcl Properties: VpcId: !Ref Vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-nacl-pro NetworkAclEntryProtectIngress: Type: AWS::EC2::NetworkAclEntry Properties: CidrBlock: 0.0.0.0/0 Egress: false NetworkAclId: !Ref NetworkAclProtect Protocol: -1 RuleAction : allow RuleNumber : 100 NetworkAclEntryProtectEgress: Type: AWS::EC2::NetworkAclEntry Properties: CidrBlock: 0.0.0.0/0 Egress: true NetworkAclId: !Ref NetworkAclProtect Protocol: -1 RuleAction : allow RuleNumber : 100 NetworkAclAssocationPublicA: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: SubnetId: !Ref SubnetPublicA NetworkAclId: !Ref NetworkAclPublic NetworkAclAssocationPublicC: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: SubnetId: !Ref SubnetPublicC NetworkAclId: !Ref NetworkAclPublic NetworkAclAssocationProtectA: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: SubnetId: !Ref SubnetProtectA NetworkAclId: !Ref NetworkAclProtect NetworkAclAssocationProtectC: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: SubnetId: !Ref SubnetProtectC NetworkAclId: !Ref NetworkAclProtect # ------------------------------------------------------------# # SecurityGroup # ------------------------------------------------------------# AlbSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: VpcId: !Ref Vpc GroupName: !Sub ${SysName}-${Env}-alb-sg GroupDescription: SecurityGroup for ALB SecurityGroupIngress: - IpProtocol: tcp FromPort: 80 ToPort: 80 CidrIp: 0.0.0.0/0 Description: from Internet Tags: - Key: Name Value: !Sub ${SysName}-${Env}-alb-sg EcsSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: VpcId: !Ref Vpc GroupName: !Sub ${SysName}-${Env}-ecs-container-sg GroupDescription: SecurityGroup for ECS Task SecurityGroupIngress: - IpProtocol: tcp FromPort: 80 ToPort: 80 SourceSecurityGroupId: !Ref AlbSecurityGroup Description: from ALB - IpProtocol: tcp FromPort: 80 ToPort: 80 SourceSecurityGroupId: !Ref EcsContainerInstanceSecurityGroup Description: from ECS Container Instance Tags: - Key: Name Value: !Sub ${SysName}-${Env}-ecs-container-sg EcsContainerInstanceSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: VpcId: !Ref Vpc GroupName: !Sub ${SysName}-${Env}-ecs-container-instance-sg GroupDescription: SecurityGroup for ECS Container Instance Tags: - Key: Name Value: !Sub ${SysName}-${Env}-ecs-container-instance-sg # ------------------------------------------------------------# # ALB # ------------------------------------------------------------# Alb: Type: AWS::ElasticLoadBalancingV2::LoadBalancer Properties: Name: !Sub ${SysName}-${Env}-alb Tags: - Key: Name Value: !Sub ${SysName}-${Env}-alb Scheme: internet-facing SecurityGroups: - !Ref AlbSecurityGroup Subnets: - !Ref SubnetPublicA - !Ref SubnetPublicC TargetGroup: Type: AWS::ElasticLoadBalancingV2::TargetGroup Properties: VpcId: !Ref Vpc Name: !Sub ${SysName}-${Env}-tg Protocol: HTTP Port: 80 TargetType: ip Listener: Type: AWS::ElasticLoadBalancingV2::Listener Properties: DefaultActions: - TargetGroupArn: !Ref TargetGroup Type: forward LoadBalancerArn: !Ref Alb Port: 80 Protocol: HTTP # ------------------------------------------------------------# # IAM # ------------------------------------------------------------# EcsTaskExecutionRole: Type: AWS::IAM::Role Properties: RoleName: !Sub ${SysName}-${Env}-ecs-task-execution-role Path: / AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: ecs-tasks.amazonaws.com Action: sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy EcsTaskExecutionRoleDatadogAgentPolicy: Type: AWS::IAM::ManagedPolicy Properties: ManagedPolicyName: !Sub ${SysName}-${Env}-dd-agent-policy Path: / PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - ecs:ListClusters - ecs:ListContainerInstances - ecs:ListServices - ecs:DescribeContainerInstances Resource: - "*" Roles: - !Ref EcsTaskExecutionRole EcsContainerInstanceRole: Type: AWS::IAM::Role Properties: RoleName: !Sub ${SysName}-${Env}-ecs-ec2-role Path: "/" AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: - "ec2.amazonaws.com" Action: - "sts:AssumeRole" ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore EcsContainerInstanceProfile: Type: AWS::IAM::InstanceProfile Properties: Path: / Roles: - !Ref EcsContainerInstanceRole InstanceProfileName: !Sub ${SysName}-${Env}-ecs-ec2-role # ------------------------------------------------------------# # Cloudwatch # ------------------------------------------------------------# EcsLogGroupAPP: Type: AWS::Logs::LogGroup Properties: LogGroupName: !Sub /ecs/logs/${SysName}-${Env}-app-task EcsLogGroupDatadog: Type: AWS::Logs::LogGroup Properties: LogGroupName: !Sub /ecs/logs/${SysName}-${Env}-datadog-agent-task # ------------------------------------------------------------# # ECS # ------------------------------------------------------------# EcsCluster: Type: AWS::ECS::Cluster Properties: ClusterName: !Sub ${SysName}-${Env}-ecs-cluster EcsTaskDefinitionApp: Type: AWS::ECS::TaskDefinition Properties: Family: !Sub ${SysName}-${Env}-app-task Cpu: 256 Memory: 512 ExecutionRoleArn: !Ref EcsTaskExecutionRole NetworkMode: awsvpc RequiresCompatibilities: - EC2 ContainerDefinitions: - Name: app Image: !Ref AppContainerImageName LogConfiguration: LogDriver: awslogs Options: awslogs-group: !Ref EcsLogGroupAPP awslogs-region: !Ref AWS::Region PortMappings: - HostPort: 80 Protocol: tcp ContainerPort: 80 # Auto discovery setting DockerLabels: com.datadoghq.ad.check_names: "[\"apache\"]" com.datadoghq.ad.instances: "[{\"apache_status_url\":\"http://%%host%%/server-status?auto\"}]" com.datadoghq.ad.init_configs: "[{}]" EcsTaskDefinitionDatadog: Type: AWS::ECS::TaskDefinition Properties: Family: !Sub ${SysName}-${Env}-datadog-agent-task Volumes: - Name: docker_sock Host: SourcePath: /var/run/docker.sock - Name: cgroup Host: SourcePath: /sys/fs/cgroup/ - Name: proc Host: SourcePath: /proc/ ContainerDefinitions: - Name: datadog-agent Image: datadog/agent:latest LogConfiguration: LogDriver: awslogs Options: awslogs-group: !Ref EcsLogGroupDatadog awslogs-region: !Ref AWS::Region Cpu: 100 Memory: 512 Essential: true MountPoints: - ContainerPath: /var/run/docker.sock SourceVolume: docker_sock - ContainerPath: /host/sys/fs/cgroup SourceVolume: cgroup - ContainerPath: /host/proc SourceVolume: proc Environment: - Name: DD_API_KEY Value: !Ref DatadogApiKey - Name: DD_SITE Value: !Ref DatadogSite EcsService: Type: AWS::ECS::Service DependsOn: Listener Properties: Cluster: !Ref EcsCluster LaunchType: EC2 LoadBalancers: - TargetGroupArn: !Ref TargetGroup ContainerPort: 80 ContainerName: app SchedulingStrategy: REPLICA DesiredCount: 1 NetworkConfiguration: AwsvpcConfiguration: SecurityGroups: - !Ref EcsSecurityGroup Subnets: - !Ref SubnetProtectA - !Ref SubnetProtectC ServiceName: !Sub ${SysName}-${Env}-app TaskDefinition: !Ref EcsTaskDefinitionApp EcsServiceDatadog: Type: AWS::ECS::Service Properties: ServiceName: !Sub ${SysName}-${Env}-datadog Cluster: !Ref EcsCluster LaunchType: EC2 SchedulingStrategy: DAEMON TaskDefinition: !Ref EcsTaskDefinitionDatadog EcsInstanceLaunchConfiguration: Type: AWS::AutoScaling::LaunchConfiguration Properties: LaunchConfigurationName: !Sub ${SysName}-${Env}-lc ImageId: !Ref ContainerInstanseAMI InstanceType: !Ref ContainerInstanceType IamInstanceProfile: !Ref EcsContainerInstanceProfile EbsOptimized: true AssociatePublicIpAddress: false SecurityGroups: - !Ref EcsContainerInstanceSecurityGroup UserData: Fn::Base64: !Sub | #!/bin/bash -xe echo ECS_CLUSTER=${EcsCluster} >> /etc/ecs/ecs.config EcsInstanceAsg: Type: AWS::AutoScaling::AutoScalingGroup Properties: VPCZoneIdentifier: - !Ref SubnetProtectA - !Ref SubnetProtectC LaunchConfigurationName: !Ref EcsInstanceLaunchConfiguration MinSize: 1 MaxSize: 1 DesiredCapacity: 1 Tags: - Key: Name Value: !Sub ${SysName}-${Env}-ecs-container-instanse PropagateAtLaunch: true UpdatePolicy: AutoScalingReplacingUpdate: WillReplace: true
上記テンプレートではAWS環境の構築の他、Datadog Agentの設定、オートディスカバリー設定(該当タスクにDockerラベルの付与)を行っています。オートディスカバリーにより、特定のコンテナ上で実行されているアプリケーションを自動で識別しデータを収集することが可能になります。
今回はApacheを利用しているので、こちらに記載のラベルを付与しています。設定値は利用アプリケーションにより異なりますので、設定値については各ページを確認ください。
上記CFnテンプレートで環境を構築することで、インテグレーションにDockerおよびApacheが追加されます。 *1
▲ インテグレーション
Datadog Agentのインストールにより、AWSインテグレーション(ECS)だけでは取得できないDockerのメトリクス(docker.*)が取得されます。 Dockerがインテグレーションされることにより、ダッシュボードも作成されています。
▲ Docker ダッシュボード
Datadog Agentのインストールにより、リアルタイムモニタリングであるライブコンテナ等も確認することが可能になります。
▲ ライブコンテナ
CFnテンプレート内で、オートディスカバリーの設定を行っていますので、特定のコンテナ上で稼働しているアプリケーション(今回はApache)が自動で識別され、メトリクス(apache.*)が取得されます。
▲ Apache ダッシュボード
今回は実施しませんでしたが、Datagog Agentの追加設定によりログ収集等も可能になります。
ちなみに、Datadog Agentのインストールおよび、オートディスカバリの設定をせず、ECSインテグレーションのみの場合は、Docker、Apcheのメトリクスやライブコンテナは確認することができません。(ECSインテグレーションのみの場合は、aws.ecs*
のようなCloudWatchが出力するメトリクスのみ参照可能。)
▲ ECS ダッシュボード
ECS on Fargate
ECS on Fargateでは、サイドカーコンテナ(監視対象コンテナのタスク定義内にDatadog Agentのコンテナを追加)で監視する構成となります。
公式ドキュメントを参考に、上記構成を構築するCFnテンプレートを作成しました。
sample-ecs-fargate-datadog.yml
AWSTemplateFormatVersion: 2010-09-09 Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: Common Configuration Parameters: - SysName - Env - Label: default: ECS App Container Setting Parameters: - AppContainerImageName - Label: default: ECS Datadog Container Setting Parameters: - DatadogApiKey - DatadogSite Parameters: SysName: Type: String Default: test Env: Type: String Default: prd AllowedValues: - prd - dev AppContainerImageName: Type: String Default: xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/xxxxx:latest DatadogApiKey: Type: String NoEcho: true DatadogSite: Type: String Default: datadoghq.com Resources: # ------------------------------------------------------------# # VPC # ------------------------------------------------------------# Vpc: Type: AWS::EC2::VPC Properties: CidrBlock: 10.0.0.0/16 EnableDnsSupport: true EnableDnsHostnames: true Tags: - Key: Name Value: !Sub ${SysName}-${Env}-vpc Igw: Type: AWS::EC2::InternetGateway Properties: Tags: - Key: Name Value: !Sub ${SysName}-${Env}-igw IgwAttach: Type: AWS::EC2::VPCGatewayAttachment Properties: VpcId: !Ref Vpc InternetGatewayId: !Ref Igw SubnetPublicA: Type: AWS::EC2::Subnet Properties: VpcId: !Ref Vpc CidrBlock: 10.0.1.0/24 AvailabilityZone: ap-northeast-1a MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub ${SysName}-${Env}-subnet-pub-1a SubnetPublicC: Type: AWS::EC2::Subnet Properties: VpcId: !Ref Vpc CidrBlock: 10.0.2.0/24 AvailabilityZone: ap-northeast-1c MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub ${SysName}-${Env}-subnet-pub-1c SubnetProtectA: Type: AWS::EC2::Subnet Properties: VpcId: !Ref Vpc CidrBlock: 10.0.3.0/24 AvailabilityZone: ap-northeast-1a MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub ${SysName}-${Env}-subnet-pro-1a SubnetProtectC: Type: AWS::EC2::Subnet Properties: VpcId: !Ref Vpc CidrBlock: 10.0.4.0/24 AvailabilityZone: ap-northeast-1c MapPublicIpOnLaunch: false Tags: - Key: Name Value: !Sub ${SysName}-${Env}-subnet-pro-1c RouteTablePublic: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref Vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-rtb-pub RouteTableProtectA: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref Vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-rtb-pro-1a RouteTableProtectC: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref Vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-rtb-pro-1c RouteIgw: Type: AWS::EC2::Route Properties: RouteTableId: !Ref RouteTablePublic DestinationCidrBlock: 0.0.0.0/0 GatewayId: !Ref Igw AssocationPublicA: Type: AWS::EC2::SubnetRouteTableAssociation Properties: SubnetId: !Ref SubnetPublicA RouteTableId: !Ref RouteTablePublic AssocationPublicC: Type: AWS::EC2::SubnetRouteTableAssociation Properties: SubnetId: !Ref SubnetPublicC RouteTableId: !Ref RouteTablePublic NatGatewayEipA: Type: AWS::EC2::EIP Properties: Domain: vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-eip-ngw-1a NatGatewayEipC: Type: AWS::EC2::EIP Properties: Domain: vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-eip-ngw-1c AssocationProtectA: Type: AWS::EC2::SubnetRouteTableAssociation Properties: SubnetId: !Ref SubnetProtectA RouteTableId: !Ref RouteTableProtectA AssocationProtectC: Type: AWS::EC2::SubnetRouteTableAssociation Properties: SubnetId: !Ref SubnetProtectC RouteTableId: !Ref RouteTableProtectC NatGatewayA: Type: AWS::EC2::NatGateway Properties: SubnetId: !Ref SubnetPublicA AllocationId: !GetAtt NatGatewayEipA.AllocationId Tags: - Key: Name Value: !Sub ${SysName}-${Env}-ngw-1a NatGatewayC: Type: AWS::EC2::NatGateway Properties: SubnetId: !Ref SubnetPublicC AllocationId: !GetAtt NatGatewayEipC.AllocationId Tags: - Key: Name Value: !Sub ${SysName}-${Env}-ngw-1c NatGatewayRouteA: Type: AWS::EC2::Route Properties: RouteTableId: !Ref RouteTableProtectA DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: !Ref NatGatewayA NatGatewayRouteC: Type: AWS::EC2::Route Properties: RouteTableId: !Ref RouteTableProtectC DestinationCidrBlock: 0.0.0.0/0 NatGatewayId: !Ref NatGatewayC NetworkAclPublic: Type: AWS::EC2::NetworkAcl Properties: VpcId: !Ref Vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-nacl-pub NetworkAclEntryPublicIngress: Type: AWS::EC2::NetworkAclEntry Properties: CidrBlock: 0.0.0.0/0 Egress: false NetworkAclId: !Ref NetworkAclPublic Protocol: -1 RuleAction : allow RuleNumber : 100 NetworkAclEntryPublicEgress: Type: AWS::EC2::NetworkAclEntry Properties: CidrBlock: 0.0.0.0/0 Egress: true NetworkAclId: !Ref NetworkAclPublic Protocol: -1 RuleAction : allow RuleNumber : 100 NetworkAclProtect: Type: AWS::EC2::NetworkAcl Properties: VpcId: !Ref Vpc Tags: - Key: Name Value: !Sub ${SysName}-${Env}-nacl-pro NetworkAclEntryProtectIngress: Type: AWS::EC2::NetworkAclEntry Properties: CidrBlock: 0.0.0.0/0 Egress: false NetworkAclId: !Ref NetworkAclProtect Protocol: -1 RuleAction : allow RuleNumber : 100 NetworkAclEntryProtectEgress: Type: AWS::EC2::NetworkAclEntry Properties: CidrBlock: 0.0.0.0/0 Egress: true NetworkAclId: !Ref NetworkAclProtect Protocol: -1 RuleAction : allow RuleNumber : 100 NetworkAclAssocationPublicA: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: SubnetId: !Ref SubnetPublicA NetworkAclId: !Ref NetworkAclPublic NetworkAclAssocationPublicC: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: SubnetId: !Ref SubnetPublicC NetworkAclId: !Ref NetworkAclPublic NetworkAclAssocationProtectA: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: SubnetId: !Ref SubnetProtectA NetworkAclId: !Ref NetworkAclProtect NetworkAclAssocationProtectC: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: SubnetId: !Ref SubnetProtectC NetworkAclId: !Ref NetworkAclProtect # ------------------------------------------------------------# # SecurityGroup # ------------------------------------------------------------# AlbSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: VpcId: !Ref Vpc GroupName: !Sub ${SysName}-${Env}-alb-sg GroupDescription: SecurityGroup for ALB SecurityGroupIngress: - IpProtocol: tcp FromPort: 80 ToPort: 80 CidrIp: 0.0.0.0/0 Description: from Internet Tags: - Key: Name Value: !Sub ${SysName}-${Env}-alb-sg EcsSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: VpcId: !Ref Vpc GroupName: !Sub ${SysName}-${Env}-ecs-container-sg GroupDescription: SecurityGroup for ECS SecurityGroupIngress: - IpProtocol: tcp FromPort: 80 ToPort: 80 SourceSecurityGroupId: !Ref AlbSecurityGroup Description: from ALB Tags: - Key: Name Value: !Sub ${SysName}-${Env}-ecs-container-sg # ------------------------------------------------------------# # ALB # ------------------------------------------------------------# Alb: Type: AWS::ElasticLoadBalancingV2::LoadBalancer Properties: Name: !Sub ${SysName}-${Env}-alb Tags: - Key: Name Value: !Sub ${SysName}-${Env}-alb Scheme: internet-facing SecurityGroups: - !Ref AlbSecurityGroup Subnets: - !Ref SubnetPublicA - !Ref SubnetPublicC TargetGroup: Type: AWS::ElasticLoadBalancingV2::TargetGroup Properties: VpcId: !Ref Vpc Name: !Sub ${SysName}-${Env}-tg Protocol: HTTP Port: 80 TargetType: ip Listener: Type: AWS::ElasticLoadBalancingV2::Listener Properties: DefaultActions: - TargetGroupArn: !Ref TargetGroup Type: forward LoadBalancerArn: !Ref Alb Port: 80 Protocol: HTTP # ------------------------------------------------------------# # IAM # ------------------------------------------------------------# EcsTaskExecutionRole: Type: AWS::IAM::Role Properties: RoleName: !Sub ${SysName}-${Env}-ecs-task-execution-role Path: / AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: ecs-tasks.amazonaws.com Action: sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy # ------------------------------------------------------------# # Cloudwatch # ------------------------------------------------------------# EcsLogGroupAPP: Type: AWS::Logs::LogGroup Properties: LogGroupName: !Sub /ecs/logs/${SysName}-${Env}-app-task # ------------------------------------------------------------# # ECS # ------------------------------------------------------------# EcsCluster: Type: AWS::ECS::Cluster Properties: ClusterName: !Sub ${SysName}-${Env}-ecs-cluster EcsTaskDefinitionApp: Type: AWS::ECS::TaskDefinition Properties: Family: !Sub ${SysName}-${Env}-app-task Cpu: 256 Memory: 512 ExecutionRoleArn: !Ref EcsTaskExecutionRole NetworkMode: awsvpc RequiresCompatibilities: - FARGATE ContainerDefinitions: - Name: app Image: !Ref AppContainerImageName LogConfiguration: LogDriver: awslogs Options: awslogs-group: !Ref EcsLogGroupAPP awslogs-region: !Ref AWS::Region awslogs-stream-prefix: !Sub ${SysName} Essential: true PortMappings: - HostPort: 80 Protocol: tcp ContainerPort: 80 # Auto discovery setting DockerLabels: com.datadoghq.ad.check_names: "[\"apache\"]" com.datadoghq.ad.instances: "[{\"apache_status_url\":\"http://%%host%%/server-status?auto\"}]" com.datadoghq.ad.init_configs: "[{}]" - Name: datadog-agent Image: datadog/agent:latest LogConfiguration: LogDriver: awslogs Options: awslogs-group: !Ref EcsLogGroupAPP awslogs-region: !Ref AWS::Region awslogs-stream-prefix: !Sub ${SysName} Cpu: 10 MemoryReservation: 256 Essential: false Environment: - Name: DD_API_KEY Value: !Ref DatadogApiKey - Name: DD_SITE Value: !Ref DatadogSite - Name: ECS_FARGATE Value: true EcsService: Type: AWS::ECS::Service DependsOn: Listener Properties: Cluster: !Ref EcsCluster LaunchType: FARGATE LoadBalancers: - TargetGroupArn: !Ref TargetGroup ContainerPort: 80 ContainerName: app SchedulingStrategy: REPLICA DesiredCount: 1 NetworkConfiguration: AwsvpcConfiguration: SecurityGroups: - !Ref EcsSecurityGroup Subnets: - !Ref SubnetProtectA - !Ref SubnetProtectC ServiceName: !Sub ${SysName}-${Env}-app TaskDefinition: !Ref EcsTaskDefinitionApp
上記CFnテンプレートで環境を構築することで、インテグレーションにFargateおよびApacheが追加されます。
▲ インテグレーション
こちらのインテグレーションにより、ECSインテグレーションだけでは取得できないFargateのメトリクス(ecs.fargate.*)が取得され、ダッシュボードも作成されます。
▲ Fargate ダッシュボード
ECS on EC2時同様、オートディスカバリの設定を行っていますので、コンテナ上で稼働しているアプリケーション(今回はApache)を自動で識別し、メトリクス(apache.*)が取得されます。
▲ Apache ダッシュボード
Datadog Agentのインストールにより、ライブコンテナ等も確認することが可能になります。
▲ ライブコンテナ
さいごに
Datadog Agentをインストールすることでメトリクスの解像度の向上や、追加設定などさまざまなメリットが得られます。
今回はログ収集など追加設定を行いませんでしたので、こちらのテンプレートを利用して検証を進めたいと思います。そちらについては、また別の機会にアウトプットしたいと思います。
参考
脚注
- インテグレーションの反映に時間を要することがあったので、事前に手動でインテグレーションしておくとよさそうです。 ↩